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ABSTRACT 

The grades which foreign students receive are not 
always based on the sane criteria as the grades assigned to native 
American students. The use of standardised test scores provides a 
coamon data base from which, to evaluate the relative proficiency 
level of foreign students. This study examines the Test of English as 
a Foreign Language fTOEFL) end the English Placeaent Exaainaticn 
(EPE) , the^ two tests aost frequently used to place foreign students. 
The objectives of this study are as follows: (1)Lto deter«j^ne the 
predictive ability of the TOEFL and the EPE using first and secojid 
semester graduate grade point average and also using a grade ip a 
r aredial English course as the criterion, (2) to assess the 
concurrent validity o£ the TOEFL and the EPE, and (3) to estimate the 
magnitude rf the change in the TOEFL scores after the subjects have 
lived in an English speaking country, and taken a remedial English 
course for one semester. (BB) 



o 

ERIC 



m 



30 



; / ' 



i?eport 





m m tomm simBm M 

Tttn <» BM6LI8B AS A WOUXm 




AUTHOR is) ^^^"^ M. AiMaoni 




4 



REPORT 350 



Htrch 1974 



ERIC 



May quoted in uhole^ or in part if credit is given the source, v 

MEASUREMENT AND RESEARCH DIVISION - OF^CE OF INSTRUCTIONAL RESOURCES 
307 ENGINEERING HALL -UNI VERS I TY OF ILLINOIS 
URBANA, ILLINOIS .61801 



ASSESS^^E^rr of the proficiency in the use and understanding of 

ENGLISH BY FOREIGN STUDENTS AS MEASURED BY THE 
TEST OF ENGLISH AS A FOREIGN LANGUAGE 



Most prediction studies of native born students indicate that previous 
academic record is the best predictor of further college achievement (Bowers, 
1965; Duff 6s Aukes, 1965). However, the academic records of foreign students 
have been found to be extremely diverse in type and content depending on the 
country of origin and the institution within the country. Thus, most efforts 
to use these diverse records as predictors of academic success in American 
colleges have been unfruitful (Putman, 1961), Since the previous academic 
records were not available for a large number of students in the present study, 
no attempt was made to use the remaining available academic records. 

Howell (1966) suggested that standardized test scores could provide a 
common data base from which to evaluate the relative proficiency level of the 
entering foreign s.udents. This data could then be used for prediction purposes, 

A number of investigators have attempted to predict academic success, l*e. , 
grade-point averaf,e (CPA) of foreign students from typical American college stan- 
dardized admission tests* Pitcher and Harvey (1963) studied the predictive use of 
the Graduate Record Examination (GRE). Coffman (1963) reported the predictive 
value of the College Board Preliminary Scholastic Aptitude Test (SAT). Howell 
(1966) studied the predictive value of the scholastic aptitude and achievement 
tests. Other tests, such as the Miller Analogies Test, have been studied for their 
predictive capabilities (Hountras, 1956). Sims (1967) summarized the studies in 
this area and reported a median correlation of about .22 for verbal ability test 
scores and about .28 for quantitative ability scores. Sims further pointed out 
that these studies did not assess the foreign student's proficiency in the use 
and understanding of English. The possible effect of the foreign student's 



2. 

proficiency in English upon performance on /uucricsn admission tests appears ah 
a reasonable next otep. This line of reasoning woiuld suggest that perhaps the 
same meaning should i.ot be assigned to a low score on a scholastic achieveaent 
or aptitude test obtained by a student with good English proficiency as would 
be assigned if obtained by a student with low English proficiency. In other 
words, the proficiency of English my nioderate the relationship between scholastic 
aptitude and achlevenzent . 

Slocum (196^.) prcbented the view for the use of the Test of English as a 
Foreign Language (TOEFL) as ^ moderator variable. He stated, Such teuts as 

the SAT and GRE are most useful in the case of foreign student applicants when 
their scores are :onsidcred in the light of the TOEFL scores. This enables one 
to make allowance: if necessary In objective test scores for a handicap in 
English. On the cir.her hand, the TOEFL score could point to rejection if it were 
high (indicating little or no handicaps in English) and the objective test scores 
(not so much v^.rbaJ aptitude scores as quantitative and achievement scores) were 
low [p. 4]/» 

However, Sharon (1970) reports inconclusive results in a study which tested 
the hypothesis that TOEFL is a tcoderator variable when combined with the 
Admissions Test for Graduate Study in Business (ATGSB) to predict success of 
foreign students in graduate business school. The TOEFL added nothing to the 
predictive validity of the ATGCB when combining them in a llnerr or a nonlinea*: 
model, Schrader and Pitcher (1970) combined the Law School Admission Test (LSAT) 
and TOEFL scores to predict success In law school of foreign students. They 
reported that the combined use of the LSAT and the TOEFL In a linear regression 
model did not increase the predictive validity obta:'.ned when either test was 
used alone. 

Sharon (1971) tested TOEFL as a moderator of the GRE scores in the pre- 
diction of foreign students* CPA in graduate school. This study reports 
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e»..atl.lly ch, «8nituda .nd conflgur.tloo of th« GRfi vaUdUiea «a ver* 

found by Pitcher «d Harvey (1963) which .ere .24 betv^ GRE-V and CPA and .32 
between GRE-Q and CPA. Hae «>r. enlightening reault reported vaa that the i^nee. 
combination of the OKE-V or Q vith the TOtFl did not reeult in aiguificaatly 
higher vallditlea over thoe. obtained with the CRB-V or Q alone. One of the 
reasona augg^ated for tbia finding i. that CPA »ay be an inappropriate criterion 
for foreign studenta. For example, in «,at college, foreig. atud«,t« with inade- 
qv«te Engllah proficiency are frequently giv«. apecial remedial Ehgliah Instruction 
and, thua. «u«t cavry lighter course loads. Sharon further postulates that 
.Uowance. are often »ad. in the grades themselves to compensate for the inability 
of the foreign student to deal adequately vith the African language, concepts 
and meaning of the material presented. 

The «»tire question of the factors which contribute to the unreliability 
\. of course grades or CPA whan they are used a. .he criteria in predictiJ atudi.a 
has Wen a compounding problem for some time (Bbel, 1961) . 

Lavln (1965) discusses several other sources of variations in assi^ing 
grades. Grades are based on different criteria. Some are based on obJeUlve 
teats, others on essay-type tests. Others include credit for oral partif:ipation 
and term papers. Grades many times include a variety of subjective fact|>rs 
based on undefined criteria. CPAs also have limitations. They vary ba^ed on 
the nuHi>er of courses used in the computation; the GPA for one student mjy be 
based on four courses whereas another student's may be based on six couries. 
The course, may vary in content, m level and difficulty. The courses mjat 
certainly wUl require differing abilities and levels of proficiency. Jvln 

further stated that the (JP>j» nf j ^ . 

upis of foreign students tend to bo based on feW courses 

because as Sharon pointed out. the majority of foreiga students carry light course 
loads because they must enroll in remedial English courses. Thl. is especially 
^true in Che early semesters «til they meet the English proflclsncy requirements. 

ERIC 



Auoth^iT factor vh.tcb ?;viv ccntr ibu:*- ; th^^ unrt'!l l.abil it y of the gradtM) for 
foreign ^itii<5ents it; the 'U^Oi-.i wLlV'- ^-mde , The prad^-'f^? vhlch fcroifn <5tudi*nt« 
receive are not aluayr* h.-.^it*.' on the Bane c ritf^ric^ art cVkj rrncku^ af^f^trnea to 
na t i ve Ar-eric/in studeTU:$^ *^it:;:.an (1961) reporceci th/iC c?no-'thlrd of the f;i':ulcv 
of the Teacher* 8 Co.lie;,\e at < olumbia Unlvorslty admittt^d to Ss-^u? leniency in 
f.rading foreign students* Thm Nnc tonal Assoc i^l ion for foreign Student Affairs 
has atte.Ttrpted to diflcourap'^' courtesy grades which are asaipned as a gesture of 
international ftood will or ml replaced compensation for the student's lack of 
familiarity with the Enr.liah lanp^uape (Walton, 1967). Paraskevopoulus and 
Dremuk (1968) report no differential standarde are used for foreign students in 
the physical and biological sciences » cither in course r^qulrenxjnt or in grading 
standard at the University of Illinois at Urbana-Chatcpaign (UIUC) . In the social 
and behavio»-al sciences where the need for language aad comin\mication proficiency 
is greater, they report that a large minority of the faculty is more lenient 
in the grading of foreign students from non^-English speaking countries and use 
differential covirse requirements for foreign students. 

These potcmtial contaminates of criteria (grades or GPA) would tend to 

/ 

lower the reliability of the criteria^ thus, reducing the correlation between 
test scores and the criteria (validity). 

Most American colleges and universities use the TOEFl- scores in conjunction 
with so-called differential, locally adninistered, placement tests which, in 
coinbination, are used to place the foreign student in remedial English courses. 
These remedial courses are designed to increase the foreign students* proficiency 
in English. Studies have been conducted to determine the predictive validity 
of a nunfcer of these tests as related to GPA. Maw;ell (1965) investigated the 
comparative validities of the TOEFL and the English proficiency test used attsb 
Berkeley, Using a sample of 238 cases, the correlation of the TOEFL with GPA 
was .17, the corresponding validity of the Berkeley tests was .11. 



Hie Unlvc'r?:^! ty of W.^fihington (1966) rcpos:t+j<,i r^irul Lir vall JUy rt^iift lcntthip*:i 
lustng the TOEFL and the EngUiih proticlmuy tttst u^»cd Wa^UiinjUon. Th*:! Spear- 
rwm rank-order correi^itlon tor both teKts» was ^2h, A rt*plic/it Ion wa.^ conducted 
In whlcti ax\ Identical rar»k-or<iet corrclacion vaci obtalniHU 

Domino (1966) conductt^d a vzilldity acudy Fresno State Coll€Tt.-^e uBing the 
TOEFL and Che Coil^ge Vocabulary Test* The validity of the TOEFL \^ CPA reported 
v^js .31^ the corresponding validity of the Coilege Vocabulary Te5it was ,34. 

This sample of ^Miidit'fi appears to 5Jhow that the locally iuitniriis tered Engli^tt 
proficiency tests produce predictive validities comp/^rablo to tluit of the TOEPl. 

In Itgl^t of thiii ooncluKion, the concurrent validity of the TOEFL with the 
locally administered English plxicetm?nt and proficiency is of interest. Arc 
these locally administered tests niTirely i redundancy of the TOEFL? 

1. Upshur (1966) reports a concurrent validity study using students 

from Indiana University (;; ^ 38), San FranciBCO State College (N » 50) 
and Park College (:V « 12) (total Ss ^ 100)* He reported a concurrent 
validity of .89 between the TOEFL and the Michigan Test of English 
Language Proficiency. 
2* Maxwell (1965) reported a study conducted at the University of 

California, Berkeley camrms. He found a concurrent validity correla-- 
tlon of .87 = 2 38) between the TOEFL and the locally developed 
English proficiency test at Berkeley. 

3. The American Language Institute (ALI) at Georgetown University (1966) 
reported a concurrent validity of .79 between the TOEFL score and the 
ALI test developed at Georgetown. 

4. Educational Testing Service (ETS) (1966) conducted a study at New York 
University coisparing the TOEFL with the ALI Tost of Proficiency in English 
developed at New York Unlvers-ity. This study also reports the 
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ccft\curr«nt validity of th<j TOEFL vith the Mlchlgjaai T*u«t of English, 
The correl4ition of the TOEFL toi^il acor** vitn che All total 8core 
va« {// « 166) aud w^ith the> HIchlgj&n test, tcr.a* "fjcore was ,8^ 
CV - 99). 

5. Pitcher And Ra (1967) conducted a BiKuiy cotsparlng the TOEPl acoree 
with ratings cf th<^rr^»*t vrictt*n by 3iO fcrci«,a pCuwcUC© v^u«ull«d in 
colleger 'ai>d unl vers Itic^^ throughout the United States. Each 
student wrote four the?»es, one on ftach of four asaigncd tc?pics. 
independent ratings of each chemcr were m<ide by two rat era on overall 
quality. Hater reliability of .92 for thf suits of the olght ratings 
uas obtained, A coTrclatian of .78 waa reported betveen the TX)EFL 
total score and B\xm of the eight ratings. A correlation of , 74 
was obtained between the Writing Ability Bubscore an the TOEFL and 
the sum of the eight ratings. 
Most college and university foreign student adtaission policies require a 
toeasure of the student *9 proficiency in the English language* 

Competence in the English language has been assumed to be the crucial factor 
in the success of the foreign student studying at an AmerlcOT college or tmi- 
versity. This view is perhaps based on face validity in that it Would be 
difficult to understand how a foreign student could learn in an Aioerlcan graduate 
school without adequate proficiency in the English language; to read, write and 
comprehend the material presented. 

Thus, most graduate schools reconmiend or require that their foreign students 
take the TOEFL la th^ir native country and submit the results for admission. 
Deaoription of TOEFL ' 

The TOEFL consists of five subtests which ar^: Listening Comprehension, 
English Structure, Vocabulary, Reading, Comprehension and Writing, A score for 
each student as well as a total score is provided by ETS. ^ 



The ETS 



£3«inu<jl tJi^ites thi^ foUowtng dei*cript '>t rhc TOEFL 



Listening Con:rpreh<.'n?^ Ion 1^* a tt^m o! tht' -ibtliry to ^n<htrti t/ind 



poker* Enp.l l^^h. 
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Kn>;Ili4h S?,ruv:ruTe ,1^* tev- 1 of recognition iiaui t^iMQx^ oi 



Port 



<:i vv'Cabuiary necej^sary for efficient 



r*> hiding. 

Part ^* Reading Ccn^ri*hen?; Ion Im a tent of thr* ability to re^uJ and 
uruieratand Hafr^Ilfih prr*Bf; inclucHnK, thf vibillty to tatike 
Inft-^rences and dra*»* concU;*ilony. 

Part 5 Writing Ability is a test of elective r;tyle» usage /wid 
^iictlcn in written Bngllsttr. 

The TOEFI. io sc ■ re^i LTS; part f^roroH as wfll total «core are ^ent 



CO the examinee and che ^^^iucac lonal Institutions lo which the student has applied 
for adtniBsion, 

A core speci f 1 ^ md detailed dericrlption of the TOEFL aubter^ts vrlll illus- 
trate the verbal, s'-riil?* bclne n^JiesBed. 

The Listenin^^ Comj>rehens io:; subtest consl?ur» of these parts: direct 
questions, conversation followed by questiona, and a lecture followed by quCBtions 
pertaining to its content. 

The second subtest of the TOEFL ifi Enp.iish Structure. The 20 items consiat 
of a segtnent of dialogue containing a blank and four multiple choice altetnatives-- 
the examinee fills the blank with the correct response. This subtest ie reported to 
measure tense, sequence of nouns, adjectives, adverbs, etc. 

The third subtest, 'Vocabulary, consists of two parts, A and B. Part A is of 
the fill-in-the-blank type with four alternatives provided. Part B consists of 



synonyias or definitions and four alternatives > one being correct. Part A contains 
15 Items and Part B, 25 items. 



Ihe fourth Q^ctlm of the WEfl, is Reading Coaqsi^xshenislon, This subceat i 
cmBiBtB oi & texc and quosl:iona bm^^d on the text* Hie sfcudent my rerofiid the 
text in order to answr the auisstions* ' »^ ^ * 

' Ihe £i£th siibtest is Vfrltteg Ability vhich bos tvo v^^ta* Bart A consistil 

• ■ ■■ • - " ' ^ 

of i$eatene€!a dn which four vords or phtMm ate \2nderlii?ed in each aenteinc^, 
TSNe mdi^rlixied i^ords or phrases are l«ibelad S,' *J^, and The studitnt is to 

select the cme Incorrect vord or phraae for each sentence^ l*art B cofisists of 

* ' ■ *■ 

tmcooplete oente^ices for which four altcttiati^s f^re provided* The examinee is 
to iselect the correct altcroatlve to'coa^>lete the sentence; This subtest appears 
to stress primarily ^>asic English grasEfflsar, 

The TOEFL was developed in 1963 as a cooperative project of 30 organisslCions, 
piiblic imd private. 'A council ^^t^ ke TOEFL tms established with itoers* from both 



private axtd sovetniaetvtal ae^ncies* 
testing of English language prbflci 



The purpose of the council was to direct the 
ttcy of foreign students who apply for &daii^* 
sion to United Ststes colleges and tmiversitles* In 1965, the administrative , 
functions of the TOEFL council ve re assigned to the co-sponsorship, of the college 
Brt trance Eicamtnat ion Board (CEElS) and EfS* 
Obyectivee / 

The objectives of the present study are as follows: • ' 
1* Determine the predictive validity of the TOEFL u^ing first or 
second secaester graduate GPA as the criterion. - 

2. Deterxalne the predictive vaiiditV of the En|;llsh Placeiaent 
» ' Examination <EPE>, developed at UIUC, upon the criteria of 

■ ' ' : • . . ^ 

>** first and second semester graduate GPA* -Md'' 

3. Assess the concurrent validity of the 1t)EFL and thfe EPE* ^, 

4. Assess the predictive validity of this TOEFL scores arid t|j^':^PE scores 
using grade jin a remedial English course as the cViterlon« 



5* Esti^siite the ^g^lttide of the cthar^ge in th^ TOEFl ucmm after 
^ fche b<ave llv<^d In Engiiah ^ipe4skii*g cotmtry ain^ tak€KS a 
res«dial l^gllsh- for om rnxs^nt^t^, 
CPA v^fts u»ed «ith the 'full r«ico^i.tion of ihe Inhetci^t liMtstionSv La^nlvoln 
(1967) poixxtiid o\^t rhft. r<jacricfc.lon o( range In 'graduates gtad«s* kt the-^ graduate" 
level co0fc tmch^XB give oaly A' a and 'B^B* "KhQ GPA of forel^ studeate.is 
further corstsrlttatad becauoC: saoat take. reesedlM. /Efigl^iih ,co thm^^^fr-ifiMcm 
.. load uatll their pmtictnmy 'tn Zti^s^.inH hm a p^cncrihed ctimti^* Emn \ 
IjDf the IL^t of j^hose liiiaitatlonsv grad^i$ ar^e the primry crii^erion the M-a«4ia^- 
ss^eat of'acadesiie «cbi^vet&e.?^£| th^lr t;^© :Iq predictive atucSle® 10 Jusi£ifliabl«« 
^ md Pl^mmnt Proei^&ma for Fomi^, Studmts at UIW \ 

The UiyC irequit^s the-.- foreiga ^tvdont to submit siCOT«a fof etthisr.fch© laglloh ^ 
Langtrnga laveacory (ELI) or the lOSFL* Tfeosie ati^denti^ vho obt^iued a score of ^ 
80-89 oa Che or a acorcs of ^8D-S69 on the TOEFL are requirisd to t«k«t th« 
UIOC EPE whiith isi ad^alrastered by tn^t Division of Etoglii^h m a Second Lai:isii^ge 
<ESL) , ^nose applican^o whosa sccrecs are above this ra^g^ ara pl^iced ia 400-li£V4^1 
■ eouirfies« Those applica«ta below t\il^ range are generally nofc accepccrd for admlBdion* 
The results of the EPE arV? used to mco^^nti the appi^priate pljice^nc Iti 
res^dlal coursea of Eaglish which wre establiehed to &uppl^mnt the apecific 
deficiency of the* atydcut* The placease^st procedure also le u»«d to advise 

or cohtrol the acad^ic course load the stu^ient caay carry* 

' ■ ■ ■ ' ' ' ■■ " . , • 

the WE comiBtB of four parts: 

Part 1 Stryctyr^ is a t^st of the recogrj^ltion of Eugliah gracasar ««d 
, sentet^ce structure i^i writing* A acore of 100 perfect; a 
, score of 55 Is ths lowest score acceptable for the usost el^sjaea- 
tary'Engl^'i5h cdu^oe* ' • 



A 
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f^ktt Z' Aiar^iX' "Cmspf eteasioss 4s ;a o£ fete i^tteirstcmdil^g o£ sr|^okcm 
' , ' / ' . Eesiglish at. nomtl Qfteigd*'^/ A seot^ ©£ 100 Is perfe^fei 55 is tte . ' . ' 

ossij^ed but fimilia^ topic ouicilise. lEfes sco'^s raasgs frass . 

••^ -■/ .• .j;. . , . ;v -. ' ^-^■■ 

'0'•9^ 0 mssm no ptobljE^m^ I mma wfy gcKj^, S ?0ori» 8 is .^e y _ 

Piirt'4 Pit(,?*i«ac.latioa ij® a teat of J^he ii^illt^ t^".to^«t«r^md ^d"^o bi?' 
t- wderstood oralXj^ Ihe scores ;raagc frosi i^^Sj a mtim of 

' - , ' ~-. % ' ■ ► 

■ SSL dom-not pt^o^m a eop^osite or total swif«: mi EfS m tfce tOEFI*, K^L 

Qi^tm in Itfs procedure^, Tthe ccje^osifce' scores are nofc stg^ttlftc^t feccaMO si ^ ' \' • 

lo^ scor^ 1^ one tesi: area cim reduce 'tofeal effecfci^enea©; l^heipefof^ a ^tMmt i [■ 

i^.mw^lf bsly as'good as fels lcw«st test ama'Bcom?^ Ute; maltlpi'ci cutoff 

Al.l/ ./Ss w*of« iaco^ Accepted for iftd£ais.saycm ; 

fco OItKI;:foir the- fall im. ' . " \ ' 

A total b£ 5s coaatltuted tiie" initial ^sassp^ fosf the. SiEfit^«fir 1970 

. Ihe fdllow-ap oaople vas reduced to' 44 for' khe Tehxy&Ty'-^197l posttesr, 
Ixk eobfM^ratl^ i^rith thcr Office of FoiD^ig^i AdMs&ibas^ the H^^iimss^t mA 



ltetoear<^V9iviflio&^^^ of tte Office of l^structi<miil Beaou^c^ {«DIR|_adMstl6t0ire4 

- .-V"^ -V ' " ... ' . v-,. . • ■ . : • ■ ■ V : ■■„.. \ ' - ; ' . 

tM lQWL ^nws^ .^tn^ento i0 Saptesdb«!gr of 1970; ' 

u ■ , ■ .'■ .'^ ■" . - . ^ : , ■ . ^ ^ ; . - . ; ; - . - . ■ . • : 

Office of Foi^li^ Ad£&i0sio^^ So M^Sl) the ITOEFi. acoress whl^ ■} 

fclift had pres<mt©d vlth tHeir appllcatioii Jfor ai^oiisoit^ to iJltJC^ The 

Sb pook thin WEWL their aaclve coiatriefs. I%^c inter'if^lng EJjse^ l^ 
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the tiae of adsdssicxn and tiis^ the 5s took the 1^EFL*-vh.tch waa submitted' with 
their admiBsixm application ranged £1:010 two oonths to^si3^ toontbs^ The Ss were 
tt'eated aa a whole nnd intervmiiig tim^ was not contiplled* 

So , . ■ ■ 

the 5^ In the study Chen proceede l with the usual piocedure for placement 
i« English courses. Six $s were dropped from the sample because they were 
iindergraduates« One hundred and ten of the remaining 148 .5g had scored below 
569 on the TOEFL submitted with their applicatita for admission which made it 
necessary for them to take the EFE examination. . 

Table 1 presents the. score nonas and claaa recommendations for tne four 
oarts of the E!?E test. The course load scale is as followsi a student ; laced 
in Rhetoric 110 or 110 ^ 400 or 401 can carry a fii^ll academic load. A atiu.^ent 
placed in Rhetoric 110 ill can carry one-half ccurse load, a student placed 
in Shetoric 109 + 110 can carry only on€^ fourth course load. The aignlficance 
and purpP.se of the rhetoric courses recommended is set forth in Table 1,/ 

The following distribution ofv students were assigned to and coBq>leted the 
prescribed rc^c^dlal Engli*5h course based on the scores they obtained on the EPE 
examination; 

English Cours e 



109 
110 
111 
114 
,115 
400 
401 



m. of sb 
2 

42 
29 

1 

1 
12 

2 



89 



The remaining 21 students TrS<v the 110 students either dropped the English 



course to which thiry were assigned or no placem<»it was made^or them by ESL* 
The 5s* scores on the EPE examination were obtained from ESL, 
The next step was to assess the impact of living and functioning in an 

English speaking environment combined with enrollment in a prescribed remedial 



English courset on the observed proficiency in English as measured by the TOEFL 
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examination. To implement this 8tep, the Office of Foreign Admissions sent a 
letter to each of the 154 Se who had taken the TOEFL examination in SeptecDber " 
1970 requesting that they participate in the research project and retake the 
TOEFL examination in February 1971, The students were given a choice of two 
administration dates in an effort to obtain maximum participation. The response 
was very disappointing; only 30 out oZ the original sanqple agreed to participate, 
an additional 14 agreed to participate when contacted by a personal telephone 
request from MARD, Thus, the posttest TOEFL was adcrinis tared to 44 5s in 
February 19 7i^ 

The remedial English (Rhetoric 110, 111, 400) course grades and first and 
second semester CPAs were obt&ined for each subject. 

The validity coefficients were determined for the TOEFL and EPE scores 
relative to the grade in the rhetoric course and first semester CPA, 

Forty-four Sa in the September 1970 TOEFL sample did not submit TOEFL* scores 
with their applications for admission. Thus, the nuisber of Sa used for comparison 
of the pre-September 1970 and Septeufcer 1970 TOEFL was reduced to 110, 

Results and Discussion 

The intercorrelations, means and standard' deviations of the TOEFL part scores 
of the initial sample are presented in Table 2. Variables l'*6 are the TOEFL scores 
which the Sa sxibmltted with their applications for admissions. Variables 7-12 are 
the TOEFL scores the Ss obtained in the September 1970 administration. 

As can be seen in Table 2, the following are the test-retest correlations 
(reliabilities) for the part scores and total score of the TOEFL. 
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Part 1 Listening Cotaprehension ,573 

Part 2 English Structure ,731 

Part 3 Vocabulary .758 

Part 4 Reading Comprehension .571 

Part 5 Writing Ability .702 

Total Score .845 



Number of Itemfl 

50 
40 
40 
30 
40 



In interpreting this table, the reader should bear in mind that the test^retest 
reliability coefficient is essentially the stability of the S^a performance rather 
than the reliability of the test. 

The differences between the means of the TOEFL part scores are presented in 
Table 3. There was an increase of nearly eight joints in the Listening Comprehension 
(p < .OS). A signifxcant difference (p < .05) was also found in the Total Score. 
Since we have no knowledge of the student *s activities during the intervening time, 
it is difficult to account for the Increase. 



TABLE 3 

Diffevenoes of the Meana for the Tb)o TOEFL Administratione 



Variables 


Pre-Septeinber 
1970 Mean 


September 
1970 Mean 


Differences 


t 


Listening 
Comprehension 


52.00 


59.98 


7.98 


9.0 


(P < 


.05) 


English 
Structure 


52.98 


53.82 


0.84 




NS 




Vocabulary 


54.90 


53.62 


-1.28 




NS 




Reading 


55.50 


53.87 


-1.68 




NS 




Writing 


53.85 


52.38 


-1.47 




NS 




Total 


539.54 


547.35 


7.81 


9.9 


(P < 


.05) 



The predictive validities (product moment correlations) of the two administra- 
tions of the TOEFL (Pre-September 1970 and September 1970) and the EPE (SepteuAer ^ 
1970) relative to first and second semester GPA criterion measure are presented 
in Table 4. The TOSFL correlations except for vocabulary 45n the September 1970 
test were significant beyond the .05 level. On the other hand, none of the EPE 
orrelations was significant. 
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TABLE A 

TOEFL and EPE Siibtest Coi\<>elation8 with First and Second Semester GPAs 

;v = 110 





1 


First 


Second 


No. 


1 Subtest 


Semes t^er 


Semester 






CPA 


CPA 


Pre-September 1970 TOEFL 


1 


Listening Comprehension 


.273 


. 303 


2 


English Structure 


.347 


.319 


3 


Vocabulary 


.183 


.279 


4 


Reading 


.179 


.226 


5 


Writing 


.390 


.340 


6 


Total 


.355 


.381 


September 1970 TOEFL 


1 


Listening Comprehension 


• 274 


.303 


2 


English Structure 


.212 


.194 


3 


Vocabulary 


• 159 


.186 


4 


Reading 


-202 


.384 


5 


Writing 


• 244 


.298 


6 


Total 


• 270 


.336 


EPE September 1970 


1 


Structure 


.034 


.039 


2 


Aural Comprehension 


.068 


.123 


3 


Conposition 


-.041 


-.041 


4 


Pronunciation 


• 008 

1 


.017 



Note.-Correlation coefficients of .16 or higher are 
significant at the .05 level. 



In general J the correlations on the pre-September measures were higher but nor 
significantly higher than those of the September measures for the TOEFL. 

It is of interest that the validities of the TOEFL Total scores are higher 
(nonsignificant) with second semester CPA than with the first semester GPA. The 
larger nur\ber of courses used to compute the second semester GPA may be a contri- 
buting factor. 

Table 5 shows the predictive validities of the September 1970 TOEFL and EPE 
part scores with grade in remedial rhetoric courses 110, 111, and 400. For Rhetoric 




" 110 and 111 the grades were Satisfactory Good, coded 4; Satisfactory Fair, coded 3; 
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Satisfactory, coded 2; Satisfactory Poor, coded 1; and Unsatisfactory , coded 0. 
In Rhetoric 400, the grades are conventional A through E for credit and were coded 
A « 5 through E = 0. Only one unsatisfactory grade was given in Rhetoric 110 and 
none in Rhetoric 111, None of the EPE subscores correlate significantly with 
grades in Rhetoric 110 or Rhetoric 400 nor is the TOEFL Total score significantly 
related to gradcj in the rhetoric courses. The Rhetoric 110 course grade is not 
significantly correlated with first semester GPA but is significantly related with 
the second semester GPA. The opposite is shown for Rhetoric 111; whereas the rhet- 
oric course grade for Rhetoric 400 is significantly correlated with both semester 
CPAs. 

TABLE 5 

TOEFLj EPE^ First and Second Semester GPAe 
V}ith Rhetoric Course Gj^ade 



No. 


Subtest 


Rhetoric Course Grade 


110^ 


111'' 


400'' 


EPE Scores September 1970 


1 

2 
3 
4 


Structure 

Aural Comprehension 
• Composition 
Pronunciation 


.026 
.089 
-.100 
-.228 


.179 
.065 
-.498* 
.195 


.259 
-.347 
-.417 

.043 


TOEFL Scores September 1970 


1 
2 
3 
4 
5 
6 


. istening Comprehension 
. .iglish Structure 

ocabulary 
Heading Comprehension 
Writing 
Total 


.079 
.303* 
-.225 
-.108 
-.503* 
-.026 


.191 
.331* 
-.060 
.211 
.174 
.231 


-.459 
-.107 
.054 
-.109 
-.326 
-.283 




Rhetoric Course Grade 






First Semester GPA 
Second Semester GPA 


.257 
.345* 


.352* 
.271 


.667* 
.679* 







40 




29 




12 



ERJC *p < .05 
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In Rhetoric 110 the TOEFL Writing subscore has a correlation of ,503 with the 
course grade. The EPE scores have lower correlations* The Aural Comprelfension 
correlation was .089# This is to be expected because the students placed in this 
course were those with low Aural Comprehension scores. The Pronunciation subscore 
of the EPE had the highest correlation, ^228 with grade in Rhetoric 110* Pronuncia- 
tion improvement is the expressed behavioral objective of Rhetoric 110« 

Rhetoric 111 shows a correlation of ,498 with the Composition subscore of the 
EPE. The behavioral objective of Rhetoric 111 is the improvement of written com- 
positions. 

The restriction of range of the EPE Composition and Pronmciation subscales 
should be taken into account in evaluating their correlations. 

Table 6 presents the multiple correlations to predict course grade in Rhetoric 
110 and 111 and first semester GPA. 



/ 

/ 

VJ 



Multiple E to Fredict Grade in Rhetoric u^id Finjt Semester 
CPA using EPE Scores ana/or TOEFL Part Scores 



Course No, 


^Criterion Grade in Rhetoric 
'"-V^y Course Predicfors 


Criterion First Semester 
GPA Predictors 






TOEFL 


EPE £. TOEFL 


EPE 6 TOEFL 


110^ 


.235 


.379* 


.703* 


. 540* 


111^ 


.573* 


.467* 


.671* 


.616* 



Note. -The TOEFL total score was not used. 
®JV « AO 



. b 



N ^ 29 
*p < .05 



The EPE (four scores) and the TOEFL (five scores) were used Separately and 
combined as predictor variables .o predict course grade in Rhetoric 110 and 111. 

Rhetoric 400 was not included because of the low For Rhetoric 110, the TOEFL 



ERs^C subscores produced the larger multiple E of .379 or compared to .235 for the EPE. 



The TOEFL total score was not Included in the subscores used in the computation 
of the multiple /?s , All of the multiple Fs are sip.nificant (p < .05) except the 
EPE scores in Rhetoric 110. 

The iticrease in the multiple /? to .703 for the combined TOEFL and EPE in pre- 
dicting the Rhetoric 110 course grade is significant (p < .OS) over the multiple R 
.379 produced when TOEFL was used alone. The increase in the-multiple J? to .671 
using the TOEFL and EPE for Rhetoric 111 is not significant over the multiple R 
•573 using the EPE subscores alone. Thus, indicating that even when using nine 
subscores (TOEFL + EPE) to predict Rhetoric 110 course grade, 49% of the variance 
can be accounted for. Rhetoric 111 presents a somewhat similar picture accounting 
for 45% of the variance when both TOEFL and EPE's nine subscores are used in com- 
puting the multiple R. 

The multiple Rs obtained using the nine subscores of the TOEFL plus EPE to 
predict the first semester GPA of the 5s in Rhetoric 110 and Rhetoric 111 were 
significant (p < .05), although accounting for only 29% of the variance In 
Rhetoric 110 and 38% of the variance in Rhetoric 111. 

Tables 7 and 8 present a summary of the stepwise multiple regression of EPE 

and TOEFL part scores (separately and combined) on grades in Rhetoric 110 and 111, 

2 

respectively. In Tables 7 and 8, the increase in R is T-egllgible after the third 
step when using the nine scores of TOEFL and EPE combined to predict .course grade. 

The three variables entered for Rhetoric 110 are (TOEFL) Writing, (TOEFL) 
English Structure and (EPE) Composition, in that order. The three variables entered 
for Rhetoric 111 are Reading Comprehension, Aural Comprehension, and Composition, 
in that order. 

Table 9 presents the test-retest reliability correlations for the 44 5s that 
took the posttest February 1971 TOEFL and the September 1970 TOEFL. The test-retest 
correlations are as fo'*lows; 

o 

ERIC 



TABLE 7 

''hemsay Table for Stepujiee Multiple Regreaaion of 
EPS and TOEFL Seoree on Grctda in Bhetoria 220 

- AO 



No. 


Variable Entcired 


Multiple 


Incziaase in 


H 








EPE Only 





1 


Pronunciation 


.228 


.052 


.032 


2 


ConpoBltlon 


.233 


.054 


.002 


3 


Aural Conprehenaloa 


.235 


.055 


.001 


4 


Structure 


.235 


.055 


.000 


TOEFL Only 


1 


Liatenlng Compr^enalon 


.280 


.078 


.078 


2 


Reading Conprehsnalon 


.374 


.140 


.062 


3 


English Structure 


.378 


.143 


.003 


4 


Writing 


.379 


.144 


.001 


5 


Vocabtilary 


.379 


.144 


.000 


TOEFL and EPE 


1 


TOEFL Writing 


.502 


.252 


.252 


2 


TOEFL English Structure 


.601 


.361 


.109 


3 


EPE Coapositlon 


.675 


.455 


.094 


4 


TOEFL Listening Conprehension 


.687 


.472 


.017 


5 


EPE Structure 


.695 


.483 


.011 


6 


EPE Proniaiclation 


.700 


.490 


.010 


7 


TOEFL Reading Conprehenaion 


.703 


.494 


.004 


8 


EPE Aural Conprehenaion 


.703 


.494 


.000 


9 


TOEFL Vocabulary 


.703 


.494 


.000 



TABLE 8 ■ "-^ 

Sumary Table for Stepjdse Multiple Re^esion of BPS and 
TOEFL Part Scoree (Separately and Conbined) on Grade in RhetoHo 111 

iV - 29 



No. 


Variable Entered 


Multiple 


Increase In 






















EPE Only 


1 


Coiqposltlon 


.498 


.238 


.238 


2 


Strticture 


.543 


.295 


.057 


3 


Pronunciation 


.566 


.320 


.025 


4 


Aural Coflq>rehensicn 


.573 


.328 


.008 




TOEFL Only 


1 


Engllch Structure 


.331 


.110 


.110 


2 


Lisftening Cosq>rehenslon 


.409 


.i67 


.057 


3 


Vocabulary 


.465 


.216 


.049 


4 


Reading Comprehension 


.467. 


.218 


.002 


5 


Writing 


.467 


.218 


.000 




'1 

TOEFL and 


EPE 






1 


TOEFL Reading Coiq>rehen8lon 


,497 


.249 


.249 


2 


EPE Aural Comprehension 


.587 


.344 


.095 


3 


EPE Cdopositlon 


.639 


.408 


.064 


4 


TOEFL Writing 


.650 


.423 


.015 


5 


EPi: Structure 


.656 


.430 


.007 


6 


TOEFL Vocabulary 


.662 


.438 


.008 


7 


TOEFL Listening Comprehension 


.668 


.446 


.008 


8 


TOEFL English Structure 


.669 


.448 


.002 


9 


EPE Pronunciation 


.671 


.450 


.002 
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Part 1 liBtenXng Coapreheaslon ,776 

Fart 2 English Structure .595 

Fart 3 Vocabulary ,7 30 

Part 4 Reading Cotspreheoaion \&06 

Part 5 Writing Ability \,763 

Total Score .822 



,T)i«ae teat-r^teat reliabliitlea are elmllar co thoae obtalrt«<S in Tabte 2 for 
th« 110 Se- 

The differences betveen the coeana of the TOEFL part acorea ar«i ptaaanted in 
Tabla 10. There vaa an increaae in all of the s&e&na. Three Increaaea %fere algnlfi- 
cjint (p < .05)^ Listening Comprehenaion, Engliah Structure and tfrltlng* Tot n-m van 
alao significant. The problexD of aaaeaalng the taeaning of the gaixi acorea vill not 
be attatapted in this paper. As is well knovn^ there are nuiaaroaa probleaw in Inter-* 
prating gain acorea (Cronbach and Parley^ 1970; Lord, 1956, 1950), Perhaps it uoald 
aaem reasonable to expect some Increaae in the acorea if the TOEFL ^loaa wMtaure pro- 
ficiency in the use of Engliah just frots the fact that the au^Jecta have beet) 
functioning in an fiogliah speaking environment frotD Septeoobar 1970 until February 
1971» Table 11 preaenits the aaoe correlation for the A4*a a« ia prea«ated In Tabie 4 
for the 110'8« The toean for first setisater vaa 4.51 and for second seaMMiter was 4,54 
vhile the atandard deviation for firat seaeater was «461 and »454 for the second 
aenester. The correlation of firat and second aeiaester CPA was »803* 
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, ' TABLE iO _ . ' ' 

^-'ffer-enees 'of ^'icms ,fc>i' the. Vre arJ Pmt .TOcFL Adrdnistmtimn 

It » 4ft 













Liscenlag Cfn^rehttmsion 


59.84 


63.41 


■ ■ 3.57 


4,0 




< 


.05) 




53.25 


57.57 


- 4.3?. 


5.3 




< 


.05) 




53.77 


'54.64 ■ ■" 


0.87. 


SS' 




* 




S4.32 




. 1.20- 


NS. 


t 






Writing AbUicy 


. 50.34 


53,02 


^ 2 ,i 68 


3,0 


<P 




.05) 


Total 


563,05 


568.31 


25,26 


2.6 
i 


<P 


< 


.05) 



' ^re Septagber 1970 

b ' 
. ^ Post « Feb'rMa'ry 1971 




CorrelatianB 1^-*^ o*^^ B)Bt WEFI, Scores, SPS 
Scor&0 with riv&t md Second Smeater GPA^'^ 





Fixat Sc»ea£cr- 
■ GPA^ 


Second Saocgtitr 
OPA 


Pretest fOEPI- 


Liseeaing CoQipreh<^aslon 


■ -.225- - 


.303 


Eagllah Structure 


.248 


.329 


Vocabulacy 


.096 


.090 


Reading Coicprehensiois 


.076 


.155 


limiting Ability 


.380 


.423 


Total 


; .253 


.321 ■ 



Listeoing CtnspreheaalOQ 


.199 - 


.262 


En^lsh. Structure 


.301 


.210 


Vocabulaicy 


.219 


.217 


Reading Gosprehensioa 


.057 


.061 


Writing Ability 


■ v,,»258 


.325 


Total 


" .259. 


.275 



Septes^ber 1970 KPE 



Structure 


.096 


.166 


Aural Cosq^rehensioti 


.236 


,346 


Coepoaition 


-.074 


-.091 


Pronuaclatioii ^ 


. ,224 


.223 
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■ 25.. ■ 

.'. ■ ' * 

In InteirpretiBg theae tables (10 and 4> the reader should bear In mlsid the 
|K)Kslble motivaciodal problem,, since tnotivatlon is an liz^ortant £actor« 

Zn both the Septes^er 1970 and Febru*Ty 1971 adminlstratloDs, &e otudenta were 
avare they were only taking the test as part of a reaaa ch study. Therefore, tnotiva- 
ticm or the lack of It is a factor to be considered^ The test administrators hoped 
* that the desire to do veil would igdLnimlee the effect of the lack of Interest or 
relnforcecient value of the scores* PurtheYaoret the TOEFL Is a IwQg test requiring 
140 minuteii so fatigue is a probable factor> Finally^ since the inttarval between 
the test and the reteat was long (five loontha) error of loeaiiFurement should not be 
confused with real changes in the students* ability as a result of learning. 

The Interconrelatlon of the TOEFL and the EPE subscores obtained in September 
1970 are presented in Tabl^ 12, 

As mij^t be oxp<2Cted^ the BPE Structure aubscore correlates *591 with the English 
Structure aubscore of the TOEFL • The EPE Aural Cotnprehension atAscore also correlates 
«510 with the EPE Structure 9di>score« The intercorrelations of the four EPE subscores 
are all significant (p < ,05) • The Intercorrelations of the five TOEFL' sdbscorea are 
also ail significant (p < .05). 

Table 13 presents the Intercorrelations and test statlatics of thc^ TOEFL test 
parts administered Septe«^er 1970 for the original sample of 154 subjects. Neither . 
Table 9 nor 10 agree with the ETS manual (1970) %rt\ich states the" ^'Listening Cowpre- 
benslon is iseasuring sotste aspect of English proficiency different from that laeaaured 
by the other four parts^ since the correlations of the Listening Conprehensicm with 

of the others are the lowest coefficients/' ETS staintains that "each of the 
parts contributes sossethlng mique to the total, however, is evidenced by the fact 
that none of the correlations between the part scores'^ as high as the reliabilities 
of the part scores.** 

o ■ ■ 
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TABLE 13 

Intercorrelations and Test Statistioa of the TOEFL 
Teat Part Saores for the September 1970 Administration 

N ■= 154 



Subtest: 


Variable 
Nuniber 


1 


2 


3 




5 


Listening Con5)rehen8ion 


1 












Ehglish Structure 


2 


.396 










VocabulaTy 




.453 


.520 








Reading Coiiq>rehension 


4 


.579 


.447 


.488 






Writing Ability 


5 


.549 


.744 


.692 


.532 




Total 


6 


.725 


.781 


.798 


.768 


.877 



To sunauarize , the predictive validities relative to GPA (product-moment 
correlations) of the two administrations of the TOEFL (Pre-September 1970 and 
September 1970) except for Vocabulary on the September 1970 test were significant 
beyond the .05 level. Whereas none of the EPE subscore correlations with GFA were 
significant. 

The predictive validities of the TOEFL Total scores were higher (nonsignif icantly) 
with second semester GPA than with the first semester CPA. The possible reason is the 
increased number of courses used to compute the second semester GPA. 

None of the EPE subscore correlations with grades in Rhetoric 110 or 400 were 
significant. Furthermore, the TOEFL Total score correlations with grades in the 
Rhetoric course were not significantly related. 

The Rhetoric 110 course grade is not significantly related with the first semester 
GPA but is significantly correlated with the second semester GPA. The relationship 
for Rhetoric 111 is reversed, but the Rhetoric 400 course grade Is significantly 
correlated to the GPA of both first and second semester. 

All of ihe multiple Ra' were significant (p < .05) except the EPE scores in 
Rhetoric 110. The increase in the multiple R to .703 for the combined TOEFL and EPE 
in predicting the Rhetoric 110 coarse grade is significant (p < .05) over the multiple 
R produced when TOEFL is used alone. This would suggest that the EPE scores are con- 
tributlJig some unique variance not present in the TOEFL scores. The multiple i?s to 
predict first semester GPA using the EPE and TOEFL are significant for those students 
in Rhetoric 110 and 111. 
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Summary and Conclusions 

The predictive validities of the TOEFL appear to be similar to the predictive 
validities for native American students using admissions test scores as reported by 
Bowers (1965) ♦ This is not to suggest that we should expect similar predictive validity 
levels txsing different predictors but only that it seems very optimistic to expect to 
be able to predict academic success for foreign, students at a higher level. thto we are 
able to predict for native American students. Chase (1972) suggests that the correla- 
tions of the TOEFL test parts with the criteria of grades or CPA might be improved if 
the subtests measured skills that were more highly related to those skills that the 
student would tise In his college work. 

To Illustrate, let us discuss the probable skills tested in each subtest of the 
TOEFL separately. The Listening Comprehension Test, parts 1 and 2, are general in 
content and only slightly tap the verbal skills that measure the student's ability to 
function in the American culture* Furthermore, the first part concentrates on personal 
responses by using 16 American given names in 20 Items. This type of response has 
little generality to the skills of the student. The third part of this subtest, which 
is a lecture » more adequately taps the skills used by the students. If the intention 
was to measure the understanding of English, Chase would suggest that this test part 
is measuring the student's ability to take notes and recall details, general intelli- 
gence, or the student* s previous knowledge of the topic of the lecture and not the 
understanding of spoken English. However, the understanding of spoken English for the 
student Is an important skill. 

The English Structure subtest deals with tense, sequence of nouns and adjectives; 
the rational for the selection of the Structure Included in the test seems obscure. 
The formal rules of grammar are clearly Important in comaaaunication, however, we are 
all aware that much communication is carried on without loss of content or meaning 
even when these formal rules are violated. Chase concludes, that the language behavior 
required by this subtest does not simulate behavior required by tue foreign student In 
O 1 American college. 
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Part 3, the Vocabulary subtest, is a rather meager sample of the total vocabulary 
needed by the student. Furthermore, this part correlates .72 with the English 
Structure subtest. This is not surprising considering the obvious overlap in format 
of both subtests. Chase suggests that a better format could be developed for this 
subtest, possibly selecting synonyms which wruld provide a more adequate sampling of 
vocabtilary, without the burden of sentence reading. Chase further points out that 
there a^re many word counts available which could be used to produce a more useful and 
meaningful test by selecting words based on graded difficulty and abstractness. 

The fourth part, Reading Comprehension, includes a short text and questions based 

on the text. In this part of the test. Chase concludes that the skills required for 

\ 

this test, namely to scan the questions and skim the text for the answers, are impor- 
tant to the student in college work. 

The fifth subtest, Writing Ability, is made up of two parts. The first. Part A, 
contains sentences with four words or phrases underlined in each sentence* The student 
is to select the incorrect underlined word or phrase. Part B is made up of incomplete 
sentences with four options presented for tlie completion of each sentence. Unfortun- 
ately, this part of the test is again bound by the formal rules of basic grammar, which 
does not really indicate the student's understanding of the ideas and concepts 
conveyed. Furthermore, this subtest correlates in the .70*s with several of the other, 
subtests. Chase concludes again that this test does not measure a skill which the stu- 
dent will use in his college activities. Furthermore, the ability to recognize 
inconsistency in grammatical forms does not really indicate how the student will 

perform in his own writing. Whereas the student's ability to write and convey Ideas 

\ 

and concepts will be used by college instructors to determine in part his course grade. 
The ability to write effectively is probably reflected to some extent in most of the 
course requirement and thus, course grades 1 Chase suggests that a more adequate writ- 
ing sample format would reflect more closely the writing skills required of the student 
^n his course work in college. 
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The f ollowing is highly speculative and is based on a limited saicple of anec- 
dotal input. Foreign students appear to draw a distinction between English and 
American English, They convey that their difficulty with the TOEFL was that it 
measures American English, formal grammar sentence structure, and deals little 

with the ability to understand and communicate in the language. This is not to 

I 

say that formal grammar ia not important in communication but, as was mentioned 
earlier 5 communication can be carried on without absolute adherence to the formal 
rules of Axoerican grammar . 

Furthermore, some foreign students state that they feel the TOEFL is culturall^y 

! 

bound by using formats such as the personal responses of the first part of the \ 
Listening Comprehension subtest. ^This may be a valid criticism of the TOEFL and 
may be limiting the correlation with GPA or grades. Thus, indicating the foreign 
student can perform successfui(|l)j^n an American college without strict adherence 
to formal rules of grammar and structure of American English. 

In conclusion, the TOEFL appears to be no better or worse than those admission 
tests used to predict success for native American students • Perhaps it could be 
improved by measuring more of the skills the foreign student will be required to 
use in his daily academic performance. 
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